57 research outputs found
Textile Taxonomy and Classification Using Pulling and Twisting
Identification of textile properties is an important milestone toward
advanced robotic manipulation tasks that consider interaction with clothing
items such as assisted dressing, laundry folding, automated sewing, textile
recycling and reusing. Despite the abundance of work considering this class of
deformable objects, many open problems remain. These relate to the choice and
modelling of the sensory feedback as well as the control and planning of the
interaction and manipulation strategies. Most importantly, there is no
structured approach for studying and assessing different approaches that may
bridge the gap between the robotics community and textile production industry.
To this end, we outline a textile taxonomy considering fiber types and
production methods, commonly used in textile industry. We devise datasets
according to the taxonomy, and study how robotic actions, such as pulling and
twisting of the textile samples, can be used for the classification. We also
provide important insights from the perspective of visualization and
interpretability of the gathered data
Ensemble Latent Space Roadmap for Improved Robustness in Visual Action Planning
Planning in learned latent spaces helps to decrease the dimensionality of raw
observations. In this work, we propose to leverage the ensemble paradigm to
enhance the robustness of latent planning systems. We rely on our Latent Space
Roadmap (LSR) framework, which builds a graph in a learned structured latent
space to perform planning. Given multiple LSR framework instances, that differ
either on their latent spaces or on the parameters for constructing the graph,
we use the action information as well as the embedded nodes of the produced
plans to define similarity measures. These are then utilized to select the most
promising plans. We validate the performance of our Ensemble LSR (ENS-LSR) on
simulated box stacking and grape harvesting tasks as well as on a real-world
robotic T-shirt folding experiment
Augment-Connect-Explore: a Paradigm for Visual Action Planning with Data Scarcity
Visual action planning particularly excels in applications where the state of
the system cannot be computed explicitly, such as manipulation of deformable
objects, as it enables planning directly from raw images. Even though the field
has been significantly accelerated by deep learning techniques, a crucial
requirement for their success is the availability of a large amount of data. In
this work, we propose the Augment-Connect-Explore (ACE) paradigm to enable
visual action planning in cases of data scarcity.
We build upon the Latent Space Roadmap (LSR) framework which performs
planning with a graph built in a low dimensional latent space. In particular,
ACE is used to i) Augment the available training dataset by autonomously
creating new pairs of datapoints, ii) create new unobserved Connections among
representations of states in the latent graph, and iii) Explore new regions of
the latent space in a targeted manner. We validate the proposed approach on
both simulated box stacking and real-world folding task showing the
applicability for rigid and deformable object manipulation tasks, respectively
Enabling Robot Manipulation of Soft and Rigid Objects with Vision-based Tactile Sensors
Endowing robots with tactile capabilities opens up new possibilities for
their interaction with the environment, including the ability to handle fragile
and/or soft objects. In this work, we equip the robot gripper with low-cost
vision-based tactile sensors and propose a manipulation algorithm that adapts
to both rigid and soft objects without requiring any knowledge of their
properties. The algorithm relies on a touch and slip detection method, which
considers the variation in the tactile images with respect to reference ones.
We validate the approach on seven different objects, with different properties
in terms of rigidity and fragility, to perform unplugging and lifting tasks.
Furthermore, to enhance applicability, we combine the manipulation algorithm
with a grasp sampler for the task of finding and picking a grape from a bunch
without damaging~it.Comment: Published in IEEE International Conference on Automation Science and
Engineering (CASE2023
EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
We study the problem of learning graph dynamics of deformable objects which
generalize to unknown physical properties. In particular, we leverage a latent
representation of elastic physical properties of cloth-like deformable objects
which we explore through a pulling interaction. We propose EDO-Net (Elastic
Deformable Object - Net), a model trained in a self-supervised fashion on a
large variety of samples with different elastic properties. EDO-Net jointly
learns an adaptation module, responsible for extracting a latent representation
of the physical properties of the object, and a forward-dynamics module, which
leverages the latent representation to predict future states of cloth-like
objects, represented as graphs. We evaluate EDO-Net both in simulation and real
world, assessing its capabilities of: 1) generalizing to unknown physical
properties of cloth-like deformable objects, 2) transferring the learned
representation to new downstream tasks
Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation
We present a framework for visual action planning of complex manipulation
tasks with high-dimensional state spaces such as manipulation of deformable
objects. Planning is performed in a low-dimensional latent state space that
embeds images. We define and implement a Latent Space Roadmap (LSR) which is a
graph-based structure that globally captures the latent system dynamics. Our
framework consists of two main components: a Visual Foresight Module (VFM) that
generates a visual plan as a sequence of images, and an Action Proposal Network
(APN) that predicts the actions between them. We show the effectiveness of the
method on a simulated box stacking task as well as a T-shirt folding task
performed with a real robot.Comment: Project website: https://visual-action-planning.github.io/lsr
Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmap
We present a framework for visual action planning of complex manipulation
tasks with high-dimensional state spaces, focusing on manipulation of
deformable objects. We propose a Latent Space Roadmap (LSR) for task planning,
a graph-based structure capturing globally the system dynamics in a
low-dimensional latent space. Our framework consists of three parts: (1) a
Mapping Module (MM) that maps observations, given in the form of images, into a
structured latent space extracting the respective states, that generates
observations from the latent states, (2) the LSR which builds and connects
clusters containing similar states in order to find the latent plans between
start and goal states extracted by MM, and (3) the Action Proposal Module that
complements the latent plan found by the LSR with the corresponding actions. We
present a thorough investigation of our framework on two simulated box stacking
tasks and a folding task executed on a real robot
Elastic Context: Encoding Elasticity for Data-driven Models of Textiles
Physical interaction with textiles, such as assistive dressing, relies on
advanced dextreous capabilities. The underlying complexity in textile behavior
when being pulled and stretched, is due to both the yarn material properties
and the textile construction technique. Today, there are no commonly adopted
and annotated datasets on which the various interaction or property
identification methods are assessed. One important property that affects the
interaction is material elasticity that results from both the yarn material and
construction technique: these two are intertwined and, if not known a-priori,
almost impossible to identify through sensing commonly available on robotic
platforms. We introduce Elastic Context (EC), a concept that integrates various
properties that affect elastic behavior, to enable a more effective physical
interaction with textiles. The definition of EC relies on stress/strain curves
commonly used in textile engineering, which we reformulated for robotic
applications. We employ EC using Graph Neural Network (GNN) to learn
generalized elastic behaviors of textiles. Furthermore, we explore the effect
the dimension of the EC has on accurate force modeling of non-linear real-world
elastic behaviors, highlighting the challenges of current robotic setups to
sense textile properties
Benchmarking bimanual cloth manipulation
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Cloth manipulation is a challenging task that, despite its importance, has received relatively little attention compared to rigid object manipulation. In this paper, we provide three benchmarks for evaluation and comparison of different approaches towards three basic tasks in cloth manipulation: spreading a tablecloth over a table, folding a towel, and dressing. The tasks can be executed on any bimanual robotic platform and the objects involved in the tasks are standardized and easy to acquire. We provide several complexity levels for each task, and describe the quality measures to evaluate task execution. Furthermore, we provide baseline solutions for all the tasks and evaluate them according to the proposed metrics.Peer ReviewedPostprint (author's final draft
- …